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Abstract  Adaptive  arrays  and  tracking  share  many  concepts,  mathematical 
tools,  practical  issues,  and  algorithms.  For  example,  ill-conditioning  of  the  sample 
covariance  matrix  for  adaptive  arrays  and  ill-conditioning  of  the  covariance  matrix 
in  a  Kalman  filter  are  both  serious  problems  that  can  be  mitigated  by  the  same 
set  of  about  a  dozen  methods,  including  Tychonoff  regularization  (called  "diagonal 
loading"  in  adaptive  arrays),  factorization  of  the  covariance  matrix,  using  principal 
coordinates  or  approximately  principal  coordinates,  etc.  The  basic  mathematics  of 
Kalman  filters  and  adaptive  arrays  includes  linear  algebra  and  probability  theory,  but 
more  specifically,  Kalman  filters  and  adaptive  arrays  use  essentially  the  same  matrix 
inversion  formula. 

Multiple  hypothesis  tracking  is  the  method  of  choice  nowadays  in  tracking,  and  it 
could  be  applied  to  adaptive  arrays  and  STAP  for  different  types  of  jamming,  clutter, 
and  targets.  In  particular,  there  is  no  reason  these  days  to  settle  for  only  one  adaptive 
antenna  pattern,  but  rather  we  could  have  a  bank  of  ten  or  more  such  adaptive 
antenna  patterns  for  the  same  batch  of  data  combined  adaptively  using  standard 
Bayesian  methods.  Sample  covariance  matrix  estimation  in  adaptive  arrays  could 
benefit  from  robust  multiple  hypothesis  algorithms  developed  for  tracking.  Adaptive 
array  design  and  STAP  can  be  viewed  as  nonlinear  estimation  problems,  which 
suggest  that  adaptive  arrays  could  profit  by  using  the  powerful  and  elegant  exponential 
family  of  probability  densities,  which  includes  the  multivariate  Gaussian  density  as 
a  special  case.  In  particular,  exact  nonlinear  filters,  which  generalize  the  Kalman 
filter,  have  been  developed  using  the  exponential  family  over  the  last  two  decades. 
Markov  random  fields,  which  are  relevant  to  adaptive  arrays,  also  correspond  to  the 
exponential  family,  according  to  the  Hammersley-Clifford  theorem. 

Tracking  and  adaptive  arrays  share  a  long  list  of  common  problems  and  solutions: 
Cramer-Rao  bounds,  fast  real-time  algorithms,  optimal  waveform  (and/or  array) 
design,  multipath,  unresolved  targets,  uncertainty  in  covariance  matrices,  non- 
Gaussian  errors,  nonlinearities,  etc.  In  all  of  these  areas,  there  is  potential  for  cross¬ 
fertilization  beyond  adaptive  array  experts  casually  reading  the  tracking  literature,  and 
beyond  tracking  experts  occasionally  attending  ASAP.  The  adaptive  array  literature 
is  focused  almost  exclusively  on  sum  patterns,  but  angle  measurements,  using 
adaptive  monopulse  antenna  patterns,  are  crucial  for  many  tracking  applications. 
New  algorithms  developed  by  tracking  experts  for  mitigating  the  effects  of  unresolved 
targets  and  multipath  on  monopulse  can  be  applied  profitably  to  adaptive  arrays. 
A  novel  algorithm  developed  by  tracking  experts  that  significantly  enhances  the 
probability  of  detection  of  targets  using  monopulse  data  (combined  with  the  usual 
sum  pattern  data)  can  also  be  applied  to  adaptive  arrays. 
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Log  of  Condition  Number 
of  Kalman  Filter  vs.  Time 
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Mitigation  of  Ill-Conditioning 

•  Double  or  higher  precision  arithmetic 

•  Tikhonov  regularization  (aka  “ridge  analysis”) 
aka  “diagonal  loading” 

•  Symmetrization  of  covariance  matrix 

•  Covariance  factorization  (e.g.  square  root) 

•  Principal  coordinates  or  nearly  principal 
coordinates. 

•  Decoupled  or  quasi-decoupled  covariance 

•  Increase  process  noise  covariance  matrix 

•  Seven  other  dubious  methods  (see  Daum,  IEEE 
Trans.  Automatic  Control,  March  1983) 
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Data  Association 
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Example  of  Data  Association  Problem 
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Solution  to  Data  Association  Problem 
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Comparison  of  Multiple  Target  Tracking 

Algorithms  (part  1  of  3) 


Formalism  or 
Algorithm 

Time 
Horizon 
Considered 
(No.  of 
Samples) 

Number  of 
Data 

Association 

Hypotheses 

Unresolved 

Data 

Modeled  in 
Algorithm 

Relative  Performance  in 
Dense  Multiple  Target 
Environments 

Computational 

Complexity 

Unresolved 

Data 

Resolved 

Data 

Exact 

Solution 

Approximate 

Solution 

Nearest  Neighbor 

1 

1 

No 

Poor 

Poor 

Low 

Low 

Nearest  Neighbor  - 
M 

1 

1 

Yes 

Fair 

Poor 

Low 

Low 

Probabilistic  Data 
Association  (PDA) 

1 

1 

No 

Poor 

Fair 

Low 

Low 

Joint  Probabilistic 
Data  Association 
(JPDA) 

1 

1 

No 

Fair 

Good 

Exp 

Medium 

JPDAM 

1 

1 

Yes 

Good 

Good 

Exp 

Medium 

Nearest  Neighbor 
JPDA 

1 

1 

No 

Fair 

Good  to 
Excellent 

Poly 

Low 

Assignment 

1 

1 

No 

Fair 

Good  to 
Excellent 

Poly 

Medium 
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Comparison  of  Multiple  Target  Tracking 

Algorithms  (part  2  of  3) 


Formalism  or 
Algorithm 

Time 
Horizon 
Considered 
(No.  of 
Samples) 

Number  of 
Data 

Association 

Hypotheses 

Unresolved 

Data 

Modeled  in 
Algorithm 

Relative  Performance  in 
Dense  Multiple  Target 
Environments 

Computational 

Complexity 

Unresolved 

Data 

Resolved 

Data 

Exact 

Solution 

Approximate 

Solution 

Dynamic 

Programming 

(Viterbi) 

Many 

1 

No 

Poor 

Good 

Poly 

Medium 

Hough  Transform 

Many 

1 

No 

Fair 

Good 

Poly 

Medium 

Multiple  Hypothesis 
Tracking  (MHT) 

Many 

Many 

No 

Good 

Optimal 

Exp 

High 

MHT-M 

Many 

Many 

Yes 

Best 

Excellent 

Exp 

High 

Morefield 

Many 

Many 

No 

Fair 

Excellent 

Exp 

High 

Symmetric 
Measurements  EKF 

Many 

Many 

No 

? 

Good 

Poly 

High 

Symmetric 

Measurements 

Nonrecursive 

Many 

Many 

No 

? 

Excellent 

Exp 

High 
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Comparison  of  Multiple  Target  Tracking 

Algorithms  (part  3  of  3) 


Formalism  or 
Algorithm 

Time 
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Considered 
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Data 

Association 

Hypotheses 
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Data 
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Algorithm 

Relative  Performance  in 
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Computational 
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Unresolved 

Data 

Resolved 

Data 

Exact 

Solution 

Approximate 

Solution 

Branching 

Many 

Many 

No 

Fair 

Excellent 

Bounded 

Med.  to  High 

Branching  -  M 

Many 

Many 

Yes 

Good 

Excellent 

Bounded 

Med.  to  High 

Multi-Dimensional 

Assignment 

Many 

Many 

No 

Good 

Excellent 

- 

High 

Multi-Dimensional 
Assignment  -  M 

Many 

Many 

Yes 

Excellent 

Excellent 

- 

High 

Exact  N-Best 
Hypotheses 

Many 

N 

No 

Good 

Excellent 

- 

Medium 
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Multiple  Hypotheses  &  Models 


Tracking  Adaptive  Arrays 


Multiple  Kalman 
filters  combined 
adaptively  (e.g., 
IMM) 

MHT  for  data 
association  (see 
Sam  Blackman’s 
book) 


Multiple  antenna 
patterns  combined 
adaptively. 

Different  types  of 
jamming,  RFI, 
clutter,  multipath; 
number  of  closely 
spaced  targets; 
array  hardware 
health;  ambiguities 
in  range,  Doppler, 
angle,  phase,  etc. 
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Varieties  of  Nonlinear  Filters 

• 

Extended  Kalman  filters 

• 

Unscented  EKFs 

• 

Particle  filters 

• 

Quasi-Monte  Carlo  hybrid  particle  filters 

• 

Daum  filters 

• 

Non-recursive  filters  (zero  process  noise) 

• 

Non-recursive  filters  (non-zero  process  noise) 

• 

Semi-recursive  filters 

• 

Bayes’  rule  &  numerical  solution  of  the  Fokker- 
Planck  equation 

• 

Other 
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Nonlinear  filter  vs.  Kalman  filter 


G.  C.  Schmidt  AIAA  J  Guidance,  Control  &  Dynamics  March-April  1993 
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Relationship  of  Filters 


SE97:1 86 


Raytheon 

Exact  Recursive  Filters 


Filter 

Conditional  Density 

Comments 

1.  Kalman 
(1960) 

r|  =  Gaussian 

r|(x,  t)  =  exp 

'-l(x-x)TC-1(x-x)] 

2.  Benes 
(1981) 

rj  exp 

JXf(x)dx1 

dx  =  f(x)dt  + 

Pss(x)  =  exp 

dw  for  f( 

X 

2  f  f(x)dx 

x)  =  gradient  =  — 
dx 

3.  Daum 
(1986) 

ilPsas(x) 

Pss  (x)  =  steady  state  solution  of 
Fokker-Planck  equation 
a  =  constant 

4.  Daum 
(1986) 

>iqa(x,t) 

q(x,  t)  =  transient  solution  of  Fokker- 
Planck  equation 
a  =  constant 

5.  Daum 
(1986) 

r|Q(x  t) 

Q(x,  t)  =  solution  of  a  certain  PDE 

6.  New 

Nonlinear 

Filter 

p(x,t)exp[eT(x,t)y(Zk,t)] 

Exponential  Family  of  probability 
densities 

SE98-280 
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Comparison  of  New  Nonlinear  Filter  and  Kalman  Filter 


Item 

Kalman  Filter 

New  Nonlinear  Filter 

1. 

Conditional  density 

Gaussian 

Exponential  Family 

2. 

Sufficient  statistic 

x  =  conditional  mean 

P  =  covariance  matrix 

vi f  =  vector  of  dimension  M 

3. 

Prediction  equations 

dx 

—  =  Fx 
dt 

—  =  fp  +  pft  +  Q 

dt 

^  =  Ai|/  +  i|/TBi|/ 

4. 

Measurement  update 
equations 

Xk  =  xk  +  Pk  Hj  Rj^1  (zk  -  Hk  xk ) 

Pk  =  Mk  -  Mk  Hj  (Hk  Mk  Hj  +  Rk  )'1  Hk  Mk 

¥k=Vk  +c(zk-tk) 

5. 

Dimension  of  sufficient 
statistic 

n  +  n(n  +  1)/2 

M 

6. 

Dynamic  model 

linear 

nonlinear 

7. 

Measurement  model 

linear 

nonlinear 

8. 

Measurement  noise 

Gaussian 

non-Gaussian 

9. 

Measurements 

discrete  time 

discrete  time 

10. 

Dynamics 

continuous  time 

continuous  time 

11. 

Computational 

complexity 

numerical  integration  of  ODE 

numerical  integration  of 

ODE 

12. 

Approximation  method 

linearization 

finite  sum  decomposition 
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New  Nonlinear  Filter 


Measurement  Update  of  *Fk 


SE97:081 


Raytheon 

Generalized  Darmois-Koopman-Pitman  Theorem 

For  smooth  nowhere  vanishing  densities,  a  fixed  finite  dimensional 
filter  exists  if  and  only  if  the  conditional  density  is  from  an 
exponential  family: 

P(X,  t|Zk)  =  a(x,  t)b(Zk,  t)  exp  [0T  (x,  t)  y  (Zk,  t)] 

W.O.L.O.G. 

p(x,  t|Zk)  =  p(x,  t)  exp  [6T(x,  t)  v|/  (Zk,  t)] 
x  =  state  vector  (n-dimensional) 
t  =  time 

Zk  =  set  of  all  measurements  up  to  and  including  time  tk 

\|/(Zk,  t)  =  sufficient  statistic 

p(x,  t)  =  unconditional  density  of  x  at  time  t 
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Assertion  About  Particle  Filters 


“  Particle  filtering  methods  beat  the 
curse  of  dimensionality  as  the  rate  of 
convergence  is  independent  of  the 
state  dimension.” 

Dan  Crisan  &  Arnaud  Doucet 
(IEEE  Trans.  Signal  Processing  March  2002) 
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Folk  Theorem  About 
Monte  Carlo  Integration 


Assumptions: 


(1)  Integrand  is  known  exactly 

(2)  N  statistically  independent  samples 

(3)  Efficient  sampling  from  the  relevant 
probability  density 

(4)  Certain  regularity  conditions 
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Definition  of 
Dimension  Free  Error 

r  =  E[(x-x)*J(x-x)]/d 

in  which 

x  =  state  vector  to  be  estimated 

x  =  estimate  of  x 

d  =  dimension  of  x 

J  =  inverse  of  the  estimation  error 

covariance  matrix  for  the  optimal  filter 

E(.)  =  expected  value  of  (.) 

(.)*  =  transpose  of  (.) 
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Dimension  Free  Error  vs.  Number  of 

Particles  for  PV-PF 
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Quote  from  Doucet’s  Website 


QUESTION:  When  you  say  “particle  filters 
beat  the  curse  of  dimensionality,”  what  do 
you  mean? 

ANSWER:  A  rather  unfortunate 
expression  we’ve  used  ourselves  to  look 
good. 

SE02-1 97 
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Multiple  Choice  Quiz 

What  is  the  approximate  ratio  (r)  of  the 
volume  of  the  unit  sphere  to  the  volume  of 
the  smallest  cube  that  contains  it  for  d  =  10 

dimensions? 

(a) 

r  =  0.5 

(b) 

r  =  0.1 

(c) 

r  =  0.01 

(d) 

r  =  0.002 
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Another  Multiple  Choice  Quiz 

What  is  the  approximate  ratio  (r)  of  the 
volume  of  the  unit  sphere  to  the  volume  of 
the  smallest  cube  that  contains  it  for  d  =  100 

dimensions? 

(a) 

r  =  10'4 

(b) 

r  =  106 

(c) 

r  =  10'10 

(d) 

r  =  1 0'69 
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Simple  Back-of-the-Envelope  Formula 


Assumptions: 


(1)  The  filtering  problem  has  a  “vaguely  Gaussian”  conditional 
probability  density  (pdf) 

(2)  The  PF  is  designed  &  tuned  to  avoid  particle  collapse 

(3)  The  PF  is  designed  &  tuned  to  sample  the  pdf  efficiently 

(4)  The  N  Monte  Carlo  samples  are  statistically  independent 

(5)  Certain  regularity  conditions 

(6)  Computational  complexity  (per  particle)  is  on  the  order  of  d“. 
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Varieties  of  Particle  Filters 


Item 

Varieties 

1.  Proposal  density 

Extended  Kalman  filter  (Gaussian) 

Unscented  filter  (Gaussian) 

Daum  filter  (exponential  family) 

Gaussian  sums 

Sums  of  exponential  family 

Other 

2.  Sampling 

Rejection 

Naive  importance  sampling 

Metropolis-Hastings  importance  sampling 

Gibbs  sampling 

Other 

3.  Re-sampling 

Every  update 

Sparse 

Adaptive 

Other 

4.  Representation  of  conditional  density 

Particles 

Smooth  Kernels  (F.  Le  Gland,  et  al) 

Other 

5.  Generation  of  samples 

Deterministic  grid 

Monte  Carlo 

Quasi-Monte  Carlo 

Hybrid 

Other 

6.  Variance  reduction  methods 

Stratified  Sampling 

Rao-Blackwellization 

Control  variates 

Antithetic  variables 

Other 
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Quasi-Monte  Carlo  Hybrid  Particle  Filter 
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Quasi-Monte  Carlo  Samples 


Monte  Carlo  Samples 


(Ref:  http://www.puc-rio.br/marco.ind/quasi_mc.html) 
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Curse  of  Dimensionality  Raytheon 
and  Particle  Filters 


Bad  Estimation 
Problem 

Nice  Estimation 
Problem 

Plain  vanilla  particle 
filters 

Carefully  designed 
particle  filter  with 
bells  &  whistles 

Avoids  curse  of 
dimensionality 
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No  particles  were  harmed  in  the 
production  of  this  talk. 


